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Abstract 

This paper introduces two complexity-theoretic formulations of Bennett's logical depth: 
finite-state depth and polynomial-time depth. It is shown that for both formulations, triv- 
ial and random infinite sequences are shallow, and a slow growth law holds, implying 
that deep sequences cannot be created easily from shallow sequences. Furthermore, the 
E analogue of the halting language is shown to be polynomial-time deep, by proving a 
more general result: every language to which a nonnegligible subset of E can be reduced 
in uniform exponential time is polynomial-time deep. 

1 Introduction 

Whereas many structures found in nature are highly complex (a DNA sequence, a cell), 
some seem much simpler, either because of their complete regularity (ice), or their complete 
randomness (gas). Bennett introduced logical depth [3] to formalize computationally the dif- 
ference between complex and non-complex (trivial or random) structures. Briefly, a logically 
deep object is one with a shorter description than itself, but which requires a long time to 
compute from this short description. 

Depth is not a measure of information contained in an object, which correlates with ran- 
domness, but rather its value, or its useful information content. According to classical [18] 
or algorithmic information theory [14], the information content of a sequence is not repre- 
sentative of its value. Consider an infinite binary sequence produced by random coin tosses. 
Although the sequence contains a large amount of information in the sense that, with proba- 
bility 1, it cannot be significantly compressed, its information is not of much value, except as 
a source of input to randomized algorithms. Contrast this with the characteristic sequence 
of the halting language, access to which enables any computably enumerable language to be 
decided in linear time. From this perspective, the halting sequence is much more useful than 
a randomly generated sequence. 

Bennett's logical depth separates the sequences that are deep (i.e., that show high internal 
organization) from those that are shallow (i.e., not deep). Informally, deep sequences are those 
which contain redundancy, but in such a way that an algorithm requires extensive resources 
to exploit the redundancy (for instance, to compress or to predict the sequence). In other 
words, deep sequences are organized, but in a nontrivial way. Highly redundant sequences 
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like 00000... are shallow, because they are trivially organized. Random sequences are shallow, 
because they are completely unorganized. One of the key features of Bennett's logical depth 
is that it obeys a slow growth law [3,11]: no fast process can transform a shallow sequence into 
a deep one. Therefore a deep object can be created only through a complex, time-consuming 
process. 

Bennett [3] showed that the halting language is deep, arguing that its depth was evidence 
of its usefulness. Juedes, Lathrop, and Lutz [11] generalized this result and solidified the 
connection between usefulness and depth by proving that every weakly useful language [8] is 
deep, where a weakly useful language is one to which a nonnegligible subset of the decidable 
languages (in the sense of resource-bounded measure theory [15]) reduce in a fixed computable 
time bound. 

Unfortunately, because it is based on Kolmogorov complexity, Bennett's logical depth is 
not computable. Lathrop and Lutz [13] investigated recursive computational depth, which is 
computable, but not within any feasible time scale. Antunes, Fortnow, van Melkebeek, and 
Vinodchandran [1] investigated several polynomial-time formulations of depth as instances 
of the more general concept of computational depth obtained by considering the difference 
between variants of Kolmogorov complexity. Deep and intriguing connections were demon- 
strated between depth and average-case complexity, nonuniform circuit complexity, and effi- 
cient search for satisfying assignments to Boolean formulas. Nevertheless, some of the depth 
notions in [1] require complexity assumptions to prove the existence of deep sequences, and 
not all the depth notions obey slow growth laws. Furthermore, [1] lacks a polynomial-time 
analogue of the Juedes-Lathrop-Lutz theorem demonstrating that useful objects are neces- 
sarily deep. 

The aim of this paper is to propose a feasible depth notion that satisfies a slow growth 
law and in which deep sequences can be proven to exist. We propose two such notions: finite- 
state depth, and polynomial-time depth. Furthermore, we connect polynomial-time depth to 
usefulness in deciding languages in the complexity class E. In both cases, the definition of 
depth intuitively reflects that of Bennett's logical depth: a sequence is deep if it is redundant, 
but an algorithm requires extensive resources in order to exploit the redundancy. 

Our formulation of finite-state depth is based on the classical model of finite-state com- 
pressors and decompressors introduced by Shannon [18] and investigated by Huffman [10] 
and Ziv and Lempel [20]. Informally, a sequence is finite-state deep if given more states, a 
finite-state machine can decompress the sequence from an input significantly shorter than 
is possible with fewer states. We show that both finite-state trivial sequences (sequences 
with finite-state strong dimension [2] equal to 0) and finite-state random sequences (those 
with finite-state dimension [6] equal to 1, or equivalently normal sequences [4]) are shallow. 
Our main result in this section shows that finite-state depth obeys a slow growth law: no 
information lossless finite-state transducer can transform a finite-state shallow sequence into 
a finite-state deep sequence. We conclude the section by proving the existence of finite-state 
deep sequences. 

Our formulation of polynomial-time depth - contrary to finite-state depth - is not based 
on compression algorithms but on polynomial-time oblivious predictors. Given a language L, 
a polynomial-time oblivious predictor is a polynomial-time computable function that, given 
an input string x, predicts the probability that x £ L. Informally, L is polynomial-time deep 
if, given more time, a predictor is better able to predict membership of strings in L. We show 
that both E-trivial languages (languages in the complexity class E) and E-random languages 
are polynomial-time shallow. Our main results in this section are a slow growth law similar to 
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that for finite-state depth and logical depth, and a theorem stating that any language which 
is "useful" for quickly deciding languages in E must be polynomial-time deep. It follows that 
He, the E version of the halting language, is polynomial-time deep. 

2 Preliminaries 

N is the set of all nonnegative integers. A (finite) string is an element of {0, 1}*. An (infinite) 
sequence is an element of the Cantor space C = {0,1}°°. For a string or sequence S and 
i,j G N, S[i . . j] denotes the substring consisting of the i th through the j th bits of S, inclusive, 
and S \ n denotes 5[0 . . n — 1]. For a string x and a string or sequence S, we write x Q S to 
denote that x = S \ n for some n G N. For a string x, its length is denoted by \x\. So, s\,S2 ■ ■ ■ 
denotes the standard enumeration of the strings in {0, 1}* in lexicographical order, where 
sq = X denotes the empty string. If x,y are strings, we write x < y if \x\ < \y\ or \x\ = \y\ 
and x precedes y in alphabetical order, and x < y if x < y or x = y. 

A language is a subset of {0, 1}*. A class is a set of languages. The characteristic sequence 
of a language L is the sequence xl £ {0, 1}°°, whose n th bit is 1 if and only if s n £ L. Because 
L i— ► xl is a bijection, we will often speak of languages and sequences interchangeably, with 
it understood that the "sequence" L refers to xl, and the "language" xl refers to L. For 
n € N, we write L \ n to denote xl \ n. Given s n £ {0, 1}*, let L(s n ) = XL[n] (the value 1 if 
s n £ L, and if s n L). Let E = U ceN DTIME(2 cn ) and EXP = U ceN DTIME(2 nC ). 

Let 1 < i < j G N. The i th projection function projj : ({0, — > {0, 1}*, is given by 
projj(xi, ...,xj) = Xi. 

3 Finite-State Depth 

3.1 Finite-State Compression 

We use a model of finite-state compressors and decompressors based on finite-state transduc- 
ers, which was introduced in a similar form by Shannon [18] and investigated by Huffman [10] 
and Ziv and Lempel [20]. Kohavi [12] gives an extensive treatment of the subject. 
A finite-state transducer (FST) is a 4-tuple T = (Q, 5, v, qo), where 

• Q is a nonempty, finite set of states, 

• 5 : Q x {0, 1} — > Q is the transition function, 

• v : Q x {0, 1} — > {0, 1}* is the output function, 

• qo G Q is the initial state. 

Furthermore, we assume that every state in Q is reachable from q^. 

For all x G {0, 1}* and a G {0, 1}, define the extended transition function 5 : {0, 1}* — > Q 
by the recursion (5(A) = qo, and <5(xa) = £(£(x), a). For x G {0, 1}*, we define the output of 
T on x to be the string T(x) defined by the recursion T(A) = A, and T(xa) = T(x)i/(<5(x), a) 
for all x G {0, 1}* and a G {0, 1}. 

A FST can trivially act as an "optimal compressor" by outputting A on every transition 
arrow, but this is, of course, a useless compressor, because the input cannot be recovered. 
A FST T = (Q,S, v, qo) is information lossless (IL) if the function x i— ► (T(x),S(x)) is one- 
to-one; i.e., if the output and final state of T on input x G {0, 1}* uniquely identify x. An 
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information lossless finite-state transducer (ILFST) is a FST that is IL. We write FST to 
denote the set of all finite-state transducers, and we write ILFST to denote the set of all 
information lossless finite-state transducers. We say / : {0, 1}°° — > {0, 1}°° is FS computable 
(resp. ILFS computable) if there is a FST (resp. ILFST) T such that, for all S G {0, 1}°°, 
lim \T(S \ n)\ = oo and, for all n G N, T(S \ n) C f(S). In this case, define T(S) = f(S). 

n—>oo 

The following well-known theorem [10,12] states that the function from {0, 1}* to {0, 1}* 
computed by an ILFST can be inverted - in an approximate sense - by another ILFST. 

Theorem 3.1. For any ILFST T, there exists an ILFST T _1 and a constant c G N such 
that, for all x G {0, 1}* , x \ (\x\ - c) C T~ l {T{x)) C x. 

Corollary 3.2. For any ILFST T, there exists an ILFST T _1 such that, for all sequences 
S, T-^TCS)) = S. 

Fix some standard binary representation <jt G {0, 1}* of each FST T, and define \T\ = 
\0~T |. For all k G N, define 

FgT <fe = {T G FST : |T| < k}, 
ILFST- fc = {T G ILFST : \T\ < k}. 

Let /c G N and x G {0, 1}*. The k-FS decompression complexity (or when k is clear from 
context, FS complexity) of x is 

D£ s (x) = ^ nun { \p\ | (3T G FST^ fc ) T(p) = x } , 

i.e., the size of the smallest program p G {0, 1}* such that some fc-bit FST outputs x on input 
p. 

For a fixed A;, Dp S is a finite state analogue of Kolmogorov complexity. For any sequence 
S, define the finite-state dimension of S by 



dim FS (5) = lim liminf D ^ s ^ (3.1) 



and the finite-state strong dimension of 5 by 

Dim FS (ff) = lim lim sup I U \ (3.2) 

fc^oo n -*oo W 

Finite-state dimension and strong dimension measure the degree of finite-state randomness 
of a sequence. The above definitions are equivalent [7, 19] to several other definitions of 
finite-state dimension and strong dimension in terms of finite-state gamblers [2,6], entropy 
rates [5,20], information lossless finite-state compressors [2,6,20], and finite-state log-loss 
predictors [9]. 

Schnorr and Stimm [17] (and more explicitly, Bourke, Hitchcock, and Vinodchandran [5]) 
showed that a sequence has finite-state dimension 1 if and only if it is normal in the sense of 
Borcl [4], meaning that for all k G N, every substring of length k occurs in S with limiting 
frequency 2~ k . 

The next two lemmas show that ILFST's cannot alter the FS complexity of a string by 
very much. 
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Lemma 3.3. Let M be an ILFST. Then 



(3ci G N)(Vfc G N)(Vx G {0, 1}*) Djg Cl (M(x)) < D£ s (x). 

Proof. The proof idea of the lemma is the following. Let k, x be in the statement of the lemma, 
let p be a fc-minimal program for x, i.e. A{p) = x where A G FST^ fc , and D^ s (x) = \p\. We 
construct A' and p' for M(x). Let p' = p and let .A' be the automata which on input p' 
simulates A(p), and plugs the output into M. The size of A' is roughly the size of A plus the 
size of M, i.e. D£+ Cl (M(x)) < D| s (x), for some constant c\. More formally, let 

5a : Q A x {0, 1} -> Q A 

be the transition function of A, with 

Qc = {(*>*)! l<i<^}c(P({0,l}*)) 2 CG{AM} 

where % G {0, 1}* are the states and Sj G {0, 1}* are the corresponding output strings, and let 
$M '■ Qm x {0, 1} — ► Qm be the transition function for M. We construct 6' : Q' x {0, 1} — > <5' 
for A'. Let 

Q' = Qa* Qm x {A, M} x {0, lp x {0, lp 

where t is a constant depending on A and M. Let ((Za^a) £ Qa, (qm,sm) G Qm, s,m G 
{0, lp and 6 G {0, 1}. Define 

S'((qA,SA), (qM,SM),A,s,m,b) 

= (SaHqa, SA),b), (q M , s M ), M, X,pvoj 2 (SA({qA, SA),b))) 
S'((qA,SA), (qM,s M ),M,s,m,b) 

= ((q A ,s A ),o~M((qM,SM),m), A, pro ) 2 (5 M ((qM, sm), rn)),X). 

□ 

Lemma 3.4. Let M be an ILFST. Then 

(3c 2 G N)(Vife G N)(Vx G {0, 1}*) d£+ c =(x) < D| s (M(x)). 

Proof. The proof is similar to Lemma 13.31 Let k, x be as in the statement of the lemma. 
By Theorem 13. 1} there exists an ILFST M _1 and a constant b such that for any string x, 
x \ \x\ -b C M~ 1 (M(x)) C x. 

Let p be a fc-minimal program for M(x), i.e. A(p) = M(x) where A G FST- fc , and 
Dp S (M(x)) = \p\. We construct A' and p 1 for x. Let y = M~ 1 (M(x)), i.e. yz = x and |z| < 6. 
Let p' = p and let A' be the automata which on input p' simulates A(p), plugs the output 
into M _1 and adds z at the end of M _1 's output. The size of A' is roughly the size of A plus 
the size of M plus the size of z (which is of size at most b), i.e. Dp^ Ca (M(x)) < Dp S (x), for 
some constant c 2 . □ 

3.2 Finite-State Depth 

Intuitively, a sequence is finite-state deep if a finite state transducer, given additional states 
(or more accurately, additional bits with which to represent the transducer), can decompress 
the sequence from a significantly shorter input. 
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Definition 3.5. A sequence S is finite-state deep if 

(3a > 0)(Vfc G N)(3A/ G N)(3°°n G N) D$g(5 f n) - D$' S (S f n) > an. 
A sequence 5" is finite- state shallow if it is not finite-state deep. 

Remark. All results in this section remain true if the quantification in the definition of 
finite-state depth is changed to 

(VJfe G N)(3q > O)^' G N)(3°°n G N) D| S (,S f n) - D$ S (S \ n) > an. 

Note that any sequence deep by the former definition must be deep by the latter definition. 

Finite-state trivial and finite-state random sequences are finite-state shallow. 

Proposition 3.6. Let S G C. 

1. //DimFs(S') = 0, then S is finite-state shallow. 

2. If S is normal (i.e., i/dimps(<S l ) = 1), then S is finite- state shallow. 

Proof. Let S G C satisfy Dim FS (S') = and let a > 0. By fl3jZ]) let k G N be such that 

hmsup — 112 < a, 

i.e. (V°°n G N) D| s (5 \ n) < an. Therefore 

(W G N)(V°°n G N) D£ S (S \ n) - D|' s (5 \ n) < D^ s (5 \ n) < an. 

Since a is arbitrary, S is finite-state shallow. 

Let S G C be normal, k G N, and a > 0. Because normal sequences have finite-state 
dimension 1, 

(W G N)(V°°n G N) D|' s (5 f n) > (1 - a) n. 

Thus 

(W G N)(V°°n G N) D$g(5 f n) - D^' S (S f n) < n - (1 - a) n = an. 
Because a is arbitrary, S is finite-state shallow. □ 

Finite-state deep sequences cannot be created easily, as the following theorem shows. 
More precisely, no ILFST can transform a finite-state shallow sequence into a finite-state 
deep sequence. 

Theorem 3.7 (Finite-state slow growth law). Let S be any sequence, let f : {0, 1}°° — ► 
{0,1}°° be ILFS computable, and let S' = f(S). If S' is finite-state deep, then S is finite- 
state deep. 

Proof. Let S,S',f be as in the statement of the lemma and M be an ILFST computing /. 
Because S' is finite-state deep, 

(3a > 0)(Vfc G N)(3fc' G N)(3°°n G N) D£ S (S" \ n) - T)% S (S' \ n) > an. (3.3) 
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Let I G N and let c = max{ci,C2} where c\,C2 are the two constants in Lemmas 13.31 and 13 . 41 
Let V = k 1 + c where k' is obtained from (|3.3p with k = I + c. For all n G N, denote by m n 
the smallest integer such that M(S \ m n ) = S' \ n. Because M is IL, it cannot visit a state 
twice without outputting at least one bit, so there exists a constant j3 > such that, for all 
n G N, n > j3m n . For infinitely many n G N, 

"In J 

= D l FS (S \m n )-B%+ c (S \m n ) l' = k' + c 

> D^ S (S* I 1 m n ) - D|' S (M(5 t m n )) LemmaEl 
= D k F ~ c (S \m n )-D$ s (M(S \m n )) k = l + c 

> D FS (M(S \ m„)) - D^' S (M(5 f m n )) Lemma EES 
= D|s(5' \ n)) - B F s(S' \ n)) definition of m n 

> an by (EOj) 

> a/5m n , because M is IL 

whence S is finite-state deep. □ 

We next prove the existence of finite-state deep sequences. We require two technical 
lemmas first, which place bounds on the FS complexity of two concatenated strings. 

Lemma 3.8. (V/ G N)(Vz, y E {0,1}*) B l FS (xy) > B l FS (x) + B l FS (y) - 2 l . 

Proof. Let l,x,y be as in the statement of the lemma and suppose D l FS (xy) = \pp'\ where 
T G FST-', p,p' G {0,1}*, with T(pp') = xy, and T(p) = x. Thus D^ s (x) < \p\ and 
there exists s E {0, l}- 2 (because T has less than 2 l states) such that T(sp') = y, i.e. 
Dk s (y) <\p'\ + 2 l . Therefore 

B l FS (xy) = \p\ + \p'\ > B l FS (x) + B l FS (y) - 2 l , 

which proves the lemma. □ 

Lemma 3.9. (3c £ N)(VZ G N)(Vx, y G {0,1}*) B l + S c (xy) < 2\x\ +B l FS (y) + 2. 

Proof. Let I, x, y be as in the statement of the lemma and let p be a minimal program for y, 
i.e. Dp S (y) = |p| where A(p) = y with p G {0, 1}* and A G FST-'. Let p' = x'Olp where 
x' is x with every bit doubled and let A' G FST- Z+C where c is a constant independent of / 
be the following FST for xy: A(p') uses d(x) to output x, then upon reading 01, it outputs 
A(p). □ 

Theorem 3.10. There exists a finite- state deep sequence. 

Proof. For all r G {0, 1}*, define the FST T r = ({q }, 5, u, q ), where, for b G {0, 1}, 5(q , b) = 
qo and v(qQ,b) = r. Define the constant c' = \T r \ — \r\ (i.e., the number of extra bits beyond 
r required to describe T r \ note that this is a constant independent of r). 

We construct the finite-state deep sequence S = S1S2 ... in stages, with Si G {0, 1}* for all 
i G N. Let <j!):N->NxNbea function such that (V/s G N)(3°°j G N) <f>(j) = (k,2 2k+1 ), and 
for all i£N, proj 2 (^(i)) = 2 2proJlW(l))+1 . Let j G N and suppose the prefix S1S2 ■ ■ ■ 5j_i has 
already been constructed. Let tj-\ = \SxS2 ■ ■ ■ Sj-i\. Let (k, k') = 4>(j), so that k' = 2 2k+1 . 
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Intuitively, at stage j, we will diagonalize against k-bit FST's to make Dpg(5'i . . . Sj) 
large, while helping a particular (k' + c)-bit FST (c the constant from Lemma I3.9|) so that 
D|' s +c (5i . . . Sj) is small. 

Let Tj S {0, l} k ~ c be /c-FS-random in the sense that 

D| s (r i )>|r i |-2 fe /2. (3.4) 

Note that such a string always exists because there are at most |FST- fc | • 2\ r i\~ 2h/2 < 2l r J 
strings contradicting (|3.4p . Let tij = 12ij_i. Let 5j = rJ J// ' rj ' be Uj/|rj| consecutive copies of 
Tj. Let T = T r . as described above. Then |T| = fc'. It is clear that T outputs Sj = r u ^ on 
any input program of length Uj/\rj\. Therefore E)pg(Sj) < Uj/\rj\. Lemma 13.91 implies that 

H*+ c {Si . . . Sj) < 2\Si . . . Sj„i\ + D|' s (5,) + 2, 

D^ s +c (Si . . . Sj) < 2tj-x + + (3.5) 



whence 



Note that 



D| s (Si . . . Sj) > D$s(Si . . . Sj-i) + B^ s {Sj) - 2 k LemmaES 
>T> k FS (Sj)-2 k 

> i^tD^s^) - f^, + 2 k Lemma ES 



> u . _ ^L 2 k l 2 - ( £L + i ) 2 fc choice of r, 

> u _ ^ 2 fc+1 

|rj| 

/ 2 fc+1 ~ 



By (J33D and (1331) . 

Dp S (5'i . . . 5,) - Dp S +c (S'i . . . Sj) 

2 k+1 1 
> I 1 — 1 - 2t,-_i - 2 



2 fe+1 + 1 
ii, I 1 ) - 2t 



11 ' 

> y - 2tj_i def of jf 

1 

= i u i + 
1 

— ^(I^j'I + 15*1 • • • "Sj'-ll) def of Uj and 

= \\S 1 ...S j \. 

Because (f>(j) revisits every pair (k,k'), with k! = 2 2k+1 , for every fc, there exists k = k! + c 
such that, on infinitely many j, the above inequality holds. Hence S is finite-state deep. □ 
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4 Polynomial-Time Depth 



Because the time bound defining polynomial-time depth is in terms of the characteristic 
sequence of a language, we focus on the class E of languages decidable in time 2 c l Sn l for a 
fixed c G N, or equivalently, n c , where n is the length of the characteristic sequence of a 
language up to the string s n . 

4.1 Measure in E 

We use Lutz's measure theory for the complexity class E, which we now briefly describe. 
See [16] for more details. 

Measure on E is obtained by imposing appropriate resource bounds on a game theoretical 
characterization of the classical Lebesgue measure of subsets of C. A martingale is a function 
d : {0, 1}* — > [0, oo) such that, for every w G {0, 1}*, 

dW = g " 

We say that a martingale d succeeds on a language L if lim sup^^^ d(L \ n) = oo. Intuitively, 
d is a gambler that bets money on each successive bit of xl, doubling the money bet on the 
bit that occurs, and losing the rest. It succeeds by making unbounded money. 

A class of languages C has p-measure zero, and we write // P (C) = 0, if there is a polynomial- 
time computable martingale that succeeds on every language in C. C has measure zero in E, 
denoted fi(C\ E) =0, if C fl E has p-measure zero. A class C has p-measure one, denoted 
fi p (C) = 1, if C has p-measure zero, where C denotes the complement of C, and C has measure 
one in E, denoted \i (C\ E) = 1, if E — C has p-measure zero. We say that a language L is 
E-random if the singleton {L} does not have p-measure zero. 

Measure in E yields a size notion on the class E similar to Lebesgue measure on the Cantor 
space. Subsets of E that have p-measure zero are then "small subsets of E" ; for example, the 
singleton set {L} for any L G E. E, being the largest subset of itself, has p-measure one. 

4.2 Polynomial-Time Depth 

This section proposes a variation of depth based on polynomial-time oblivious predictors, 
which, given a language L, try to predict L[n] (i.e., the membership of s n in L), without 
having access to L[0 . . n — 1]. This is in contrast to a martingale, where the bet on L[n] is 
by definition a function of L[0 . .n — 1]. Intuitively, L is polynomial-time deep if giving a 
polynomial-time predictor more time allows it to predict bits of L with significantly greater 
accuracy. 

An oblivious predictor is a function P : {0, 1}* x {0, 1} — > [0, 1] such that, for all x G 
{0, 1}*, P(x, 0) + P(x, 1) = 1. Intuitively, when trying to predict a language L, P(x, 1) is the 
probability with which the predictor predicts that x G L. To measure how well a predictor P 
predicts L, we consider its associated martingale p : {0, 1}* — > [0, oo) given by 

p{L\n) = r H P(y,L(y)). 

y<s n 

We shall consider predictors P such that P(s n , b) is computable in time polynomial in n (hence 
computable in time 2 c l Sn ' for some constant c), and call such a P a polynomial-time oblivious 
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predictor, and we call the martingale p its polynomial-time oblivious martingale (pom), with 
the convention that predictors are given in uppercase and pom in lowercase. 



Definition 4.1. A language L is polynomial-time deep if there exists a > such that, for all 
pom p, there exists a pom p' such that, for infinitely many n G N, 

P'(L [ n) 

77 : v > alogra, 
p(L \ n) 

with the convention that ^ = 00. L is polynomial-time shallow if it is not polynomial-time 
deep. 

Languages that are trivial or random for E are polynomial-time shallow. 
Proposition 4.2. Let L be a language. 

1. If L G E, then L is polynomial-time shallow. 

2. If L is E-random, then L is polynomial-time shallow. 

Proof. For the first item, let a > and L G E. Then there exists a pom p that predicts L 
correctly on every string, i.e., p(L \ n) = 2 n for every n G N. Hence for any pom p' we have 

p'(L \ n) 2" 

< — = 1 < alogn 

— on 



p(L \n) ~2 

for all but finitely many n. Because a is arbitrary, L is polynomial-time shallow. 

For the second item let a > and L be E-random. Then for any pom p there exists c p G N 
such that for every n G N, p(L \ n) < c p . Fix a pom p such that p(L \ n) > 1 for all n G N. 
Then for any pom p' we have 

p'( l \ n ) ^ c p' , 
p(L f n) 1 

for all but finitely many n. Thus L is polynomial-time shallow. □ 



4.3 Slow Growth Law 

Let / : {0, 1}* — ► {0, 1}*. We say / is monotone if, for all x,y G {0, 1}*, x < y f(x) < 

f(y). Given / : N — > N, we say / is l-bounded if, for all x G {0, 1}*, \f{x)\ < l(\x\). Given two 
languages L\,L<2 and a time bound t : N — > N and length bound / : N — > N, we say that Li is 
t-time l-bounded monotone many-one reducible to L2 (abbreviated t-l-M reducible), and we 
write Li <J^ L2, if there is a Turing machine M computing a monotone, Z-bounded reduction 
/ : {0, 1}* — ► {0, 1}* such that, on input s n , M halts in at most = t(logn) steps and 

outputs G {0,1}* such that s n G L\ if and only if f(s n ) G L2. We say L\ is E-time 

linearly bounded monotone many-one reducible to L2 (abbreviated E- Lb- M reducible), and we 

E Lb 2 c l Sn I els 

write L\ <^ L2, if there exists c G N such that L\ < M ' n| L<i- We follow the convention 
of letting n refer to the length of a characteristic sequence, rather than the length of the input 
string s n . Therefore, equivalently, L\ <m'™ ^2; i-e., f(s n ) is computable in time n c , and, if 
m G N is such that s m = f(s n ), then m < n c . 

The following result shows that shallow sequences cannot be transformed into deep ones 
by simple processes. 
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Theorem 4.3 (Polynomial-time slow growth law). Let L±, L>2 be languages such that L\ <^j Lb 
L2. If L\ is polynomial-time deep, then L2 is polynomial-time deep. 

Proof. Let / : {0, 1}* — > {0, 1}* be the E-Lb-M reduction from L\ to L 2 , and let c G N such 
that / is computable in time n c (= 2 c l Sn l), so that, for all x G {0, 1}*, x G L\ f(x) G L2, 

and |/(x)| < c\x\. 

Let j»2 be any pom, such that P2 is computable in time n k for some k. Consider the pom 
pi, where, for all x G {0,1}* and b G {0,1}, Pi(x,b) = P 2 {f{x),b). Then, if x = s n , P\ is 
computable in time n c + < n c + ra cfc , so Pi is computable in time polynomial in n. 

Since L\ is polynomial-time deep, there exist a pom p±, a constant a > 0, and an infinite set 
iVCN such that, for every n G N, 

?%4^>alo g n. (4.1) 
pi(Li r n) 

Consider the following pom p' 2 , where, for all y G {0, 1}* and b G {0, 1}, 



P2(y,b) 



P[{f- l {y),b) if f-\y) exists, 
P2(y,b) otherwise. 



For all re G N, define m„ G N such that s mn = f(s n ). Because / is monotone, it is 1-1. Thus, 
if f~ l {y) exists, then it is unique. Because / is monotone, n < m n . Therefore, letting y = s mn 
and x = f~ 1 (y) = s n , x can be computed from y in time polynomial in m n by searching all 
strings w < s mn and checking whether f{w) = y, which takes at most m n n c < m^ +1 steps. 
Hence P2 is polynomial-time computable. 
For every n G N, 

alogn<^4^ by© 
Pl\Li \ n) 

Y[ x <s n P[^Mx)) 

Uyef({s ,...,s n })P2(y^ L 2(y)) 

= Wi — TTTT putting y = f{x) 

= Uy< f (s n )P2(y^2(y)) p£( y ,b) = p 2 (y,b) if f-\ y ) 

Uy<f( Sn ) -^(y, L 2 (y)) undefined, and / monotone 

= P2( L 2 \m n ) 
p 2 (L 2 \ m n )' 

Because / is linearly bounded, for all re G N, m n < n c . Thus for any n G N, 

a 1 ^ « 1 c 1 ^ ^2(^2 t 

— log m n < — log n = a log n < 



Thus the constant a/c testifies that L2 is polynomial-time deep. □ 
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4.4 Languages that are Useful for E 

In [3] Bennett showed that the halting language is deep, and Juedes, Lathrop, and Lutz [11] 
generalized this result by showing every weakly useful [8, 11] language is deep. We prove 
a polynomial-time version of the result of Juedes, Lathrop, and Lutz, namely, that every 
E-Lb-M weakly useful language is polynomial-time deep. 

Following the definition of weakly useful languages from [11] and [8], we define a language 
L to be E-Lb-M weakly useful if the set of languages in E that are reducible to L - within 
a fixed time and length bound - is not small (does not have measure zero in E). Intuitively, 
an E-useful language is somewhere in between an E-hard language and a trivial language, in 
the sense that the language does not necessarily enable one to decide all languages in E, but 
rather a nonnegligible subset of them. Note, however, that an E-hard (for instance, under 
polynomial-time many-one reductions) language may not necessarily be E-Lb-M weakly useful 
because of the requirements that an E-Lb-M reduction be monotone and linearly bounded. 

Definition 4.4. A language L is E-Lb-M weakly useful if there is a c € N such that the set 
of lang uages 2 c ' Sn '-c|s n |-M reducible to L does not have measure zero in E, i.e., if 



, ^2 c l s "l,c| S „| 

\x L-m 



E 



where 

c|sn 



^l 3 »l,c| S „| 
— M 



= {a\a<T^l}. 



In other words, a language L is weakly useful if a nonneglible subset of E monotonically 
many-one reduces to L within a fixed exponential time bound and fixed linear length bound. 
An example of an E-Lb-M weakly useful language is the halting language for E, defined as 
follows. Fix a standard linear-time computable invertible encoding of pairs of strings (x, y) i— > 
(x,y). Let Mi, M2, ... be an enumeration of machines deciding languages in E, where machine 
Mi runs in time 2*' Sn L The E-halting language is given by = { (0 J ,x) | M, accepts x }. 
It is easy to verify that access to the E-halting language allows one to decide every language 
Li G E, decided by machine Mj, using the 1.01 \s n [-time-bounded, 1.01|s n [-length-bounded, 

> 1.01|s n |,l-01|s„| 

monotone reduction f(x) = (0 l ,x); i.e., E C i/~ M , whence He is E-Lb-M weakly 

useful. 

For every g : N — > N and pom p define 



D° = < LeC 



(3 pom p')(3°°n G N) ^j^y > g(n) } 



Note that L is polynomial-time deep if and only if there exists a > such that, for all pom 
p, L € D p 8 . 

The next lemma shows that most languages in E are contained in Dp 1 for fixed g and p. 

Lemma 4.5. For any g : N — > N such that g{n) = o(2 n ) and any pom p, /j, (Dp \ E) = 1. 

Proof. Let g,p be as in the statement of the lemma. Let L £ E - Dp 1 . It suffices to show 
that p succeeds on L. L G E implies the existence of a pom p' such that for any n G N, 
p'(L [ n) = 2 n . L Dp implies that for all pom p' there are infinitely many n G N such that 
p(L [ n) > p'(L [ n)/g{n). Thus p(L [ n) > 2 n /g(n), which grows unboundedly as n — > 00 
because g = o(2 n ); i.e., p succeeds on L. □ 
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Theorem 4.6. Every E-Lb-M weakly useful language is polynomial-time deep. 



E J 7^ for some 



Proof. Let B be any E-Lb-M weakly useful language, i.e. [i \B 
c € N. Let a = ~ and let p2 be any pom. It suffices to show that B E Dp 2 logn . Let p\ be 



constructed from p2 as in the proof of Theorem 14.31 By Lemma 14.51 A* ( D pi 



log n 



1. Thus 



lo 2 c l Sll l ,c|s ?l | loen > 2 c l s "l,c|s n | 

Dpi n S-m ^ 0, whence there exists a language A € Z? Pl n £?-m . Thus 

A <M ISn ' ,cM ^ and A E Dp° gn , so by the proof of TheoremEU B € £>p 2 logn . □ 

Corollary 4.7. #e polynomial-time deep. 

Corollary 4.8. No language in E is E-Lb-M weakly useful. 

Corollary 4.9. No E-random language is E-Lb-M weakly useful. 

No decidable language is deep in the sense of Bennett [3] (see also [11, Corollary 5.7]). 
However, the halting language H is deep and, while not decidable, is computably enumerable. 
Compare this with the fact that Corollary 14.81 (or a simple diagonalization) implies that 
He E. It is easy to verify, however, He € DTIME(2' S ™ ) C EXP. Thus, polynomial-time 
depth mirrors Bennett's depth in that E-decidable languages are not polynomial-time deep, 
but polynomial-time deep languages can be found "close" to E. Similarly, Lemma 14.51 tells 
us, in an analogous fashion to Corollary 5.10 of [11], that "partially deep" sequences can be 
found in abundance in E. 

Acknowledgment. We thank Jim Lathrop for many useful and stimulating discussions in 
the early stages of this research. 
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